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ABSTRACT 

This paper presents research findings concerning the 
Career Ladder pilot test program in Arizona. The program is designed 
to reward and motivate teachers based on performance. One of the 
program's key features is the flexibility and innovation allowed to 
participating districts in their individual development of program 
designs and structures. An example of a district program is provided, 
followed by a theoretical model of program support and focus factors 
which influence district potential for successful implementation of 
an external educational reform program such as Career Ladders. An 
associated diagnostic profiling procedure for assessing the 
operational level of each of the focus and support factors as they 
currently exist within each district is then explained. Through a 
series of qualitative anecdotes, the viewpoints of career ladder 
program participants are presented, focusing on such areas as student 
achievement, teacher skills, administrator skills, professional 
input, funding, and organizational climate and communications. 
Exhibits include: (1) a matrix of initial legislative mandates, major 
research findings, and implications for future policy; (2) a summary 
of the anecdotal reports from participating districts; (3) a summary 
of recommendations to the Joint Legislative Committee on Career 
Ladders; and (4) an analysis of the impact of the Career Ladder 
program on student achievement. (JDD) 
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FINAL ACCUMULATIVE RESULTS AND TRANSFER OF 
KNOWLEDGE OF THE ARIZONA CAREER LADDER 
RESEARCH AND EVALUATION PROJECT 

Impact on Student Achievement, Formulated Models, Network Anecdotes, 
and Recommendations to the Legislature for Policy Development, 
Program Continuation and State-Wide Expansion 

Introduction 

Overview of Program F indings to Date 

The Arizona Career Ladder Program has provided enough time and experience to 
show considerable developmental success toward meeting the program goals related to 
rewarding and motivating teachers based on performance and accounting for that 
performance with more than just the processes of evaluating teachers' classroom 
methodologies. Districts and teachers have made substantial progress in actually 
demonstrating positive effects on student achievement through valid curricular content 
which has been planned to meet student needs in each local district and school. (See 
Exhibit D, pp. 50-52, Research Findings Related to Student Achievement). 

The second major program success relates to the extended time and additional 
responsibilities which have enlarged critical job assignments of teacher leaders. This 
program encourages teachers to move from the isolation of individual classrooms and 
empowers them to assume major responsibilities in assisting schools, districts and the total 
community in meeting the social, emotional and academic needs of students. 

Organization of the Report 

The purpose of this paper is to present the research findings to date concerning the 
Career Ladders pilot test program in Arizona. Research results will be organized according 
to the following themes. 

A key feature of the legislative guidelines is the flexibility and innovation allowed to 
participating districts in their individual development of program designs and structures. A 
sample plan is presented and its components discussed. 

The next topic consists of a proposed model of organizational focus and support 
factors which need to be operating satisfactorily for outside change programs to be 
maximally effective. This comprehensive model was developed from the researchers* 
observation of diverse individual districts' struggles to comply with theoretically "fair," 
uniform legislative guidelines, it points up the fact that the most well-intentioned 
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educational reform eff< rts are doomed to failure if existing organizational elements are not 
currently at adequate readiness levels to support such change efforts. 

The repeated observation of this model in practice has essentially validated its 
usefulness for a number of diagnostic and evaluative purposes. One application of the 
focus and support factors is sketched out relative to one another and to a zero baseline. By 
doing so, it enables users to assess, at a glance, the current level of operational health of 
each of these factors. 

Classifying districts according to their various levels of readiness has also led to a 
proposed three-tiered Career Ladder participation and placement scheme. This would 
enhance the potential for a successful experience with the program, by realistically 
replacing the "all-or-nothing" participation system with one which is more closely matched 
to actual present capabilities. 

The first exhibit contains a convenient, side-by-side columnar summary of initial 
legislative mandates concerning the Career Ladder pilot test, the major research findings to 
date, and the resultrnt implications for future policy regarding this teacher incentive 
program. These results are clustered according to certain key organizational focus and 
support factors, thereby linking them back to the proposed readiness model. 

Exhibit B contains the participating districts 1 responses to positive results of the 
Career Ladder program as reported under each organizational focus and support factor. As 
such, these qualitative, open-ended comments, or anecdotes, provide an in-depth, rich 
view of the effects of the Career Ladder program as it was actually experienced by the 
participants themselves. Responses are organized under each focus and support factor and 
summarized in tabular fashion. The accompanying narrative provides ample illustrations of 
the detailed, colorful quotes provided by participating districts. 

The third exhibit consists of an executive summary of recommendations to the Joint 
Legislative Committee on Career Ladders (JLCCL). Research findings, as well as critical 
lessons which have been learned to date, are concisely outlined. 

Finally, the fourth exhibit summarizes all of the available research findings 
concerning the impact of the Career Ladder program on student achievement. Results have 
converged on four critical conclusions with respect to student learning. These are 
diagrammed in summary fashion on page 52 of Exhibit D. 

pyngrfim pedgns and Structures 

The Arizona Career Ladder program has been structured so as to allow each district 
the optimal combination of flexibility and structure. Each participating district has been 
responsible for formulating its own plan, which documents requirements relative to 



evaluation procedures, opportunities for teacher advancement, financial and intrinsic 
incentives, and mechanisms for resolving questions and conflicts. This relative autonomy 
in creating plans allowed each district to consider its own unique operating conditions and 
to ensure that these were fully and equitably integrated into the associated "rules and 
regulations" of the external program. 

Figure 1, p. 4, shows an example developed by a district for the 1986-87 year. 
This district has four levels of placement, as well as six steps within each level, for its 
participating Career Ladder teachers. Each level has been characterize by an array of 
required icsponsibilities, as well as maximum and minimum levels of overall experience 
and teacher development activities. 

As described earlier, the mandate for accountability necessitates having a well- 
developed evaluation system in place. This district's evaluation procedures include 
submission of a portfolio with clearly spelled out- documentation of teacher activities. 
Associated review and assessment procedures for this evidence of participant activities are 
alsoincluded as a part of the evaluation description. 

Even the most carefully thought out program design is itself subject to revision, as a 
"paper plan" is tested against the reality of sometimes-unpredictable organizational 
operating conditions. In this spirit, the bottom portion of Figure 1, on the following 
page, contains a listing of participant reactions to the initial attempts at implementation of 
their plans. Review of "plan-vs.-reality" helps ensure that the internal self-monitoring and 
feedback mechanism generates timely, relevant information for fine-tuning program 
structures so as to be maximally operational. 

Legislative guidelines have mandated specific areas which need to be addressed in 
applicant-districts' program designs. These include professional advancement; education 
as a requirement; plans for program implementation; periodic program evaluation; and 
administrative evaluation. By explicitly including these areas, the intent of the legislature 
was for districts to "think through" the full range of ramifications of such an external 
incentive program, as it would impact upon their existing organizational systems. As a 
result, the plans which these districts developed for submission were expected to be 
sufficiently detailed and comprehensive. 

Evidence from the five-year pilot test indicates that the program plans submitted by 
district-applicants have, in fact, been exemplary -- on paper, at least. Implementation of 
these "elegant blueprints," however, has been another story. Individual districts' struggles 
to comply with their own well-intentioned guidelines have exemplified the diversities in 
readiness referred to in preceding sections of this document. In particular, the 
implementation plans of these carefully developed program designs and structures 
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Figure 1 



PROGRAM DESIGNS AND STRUCTURES 
Sample from One District for 1986-87 

Career Levels 

Level 1 Minimum 1 year, maximum 3 years/ Assigned to Level IH mentor/ 45 hours 
of inservice required/ Competent evaluations minimum/ Emphasis is on 
classroom teaching performance. 

Level 2 Minimum 1 year in-district; no max./ Teamed with Level III/ 15 hours in 
inservice/ One committee minimum/ Document student academic progress/ 
High level evaluations/ Emphasis on classroom perfoimance and sharing of 
expertise. 

Level 3 Minimum 5 years in-district; no max/ Mentor; Models instruction/ No 
minimum hours of inservice/ Conducts school inservice activity/ Five 
additional work days/ Outstanding evaluations/ Emphasis on classroom, 
sharing of expertise, modeling and teacher preparation. 

Level 4 Minimum 1 year level III; no max/ Leadership/ Assists all other levels/ Ten 
additional work days/ Emphasis on classroom, mentoring, teaming activities 
and teacher training. 

* Six steps within each level. 

Evaluation 

Portfolio Contents : Administrative approval and letters of recommendation; 

classroom evaluations; professional growth activity; plan of 
action for student progress; committee, special project or 
curriculum work; student academic progress; inservice 
planning and conducting; time and training reouirements met; 
comprehensive review panel results (level 4). 

Procedures : Evaluated by administrators; level 1 (3-4 observations)/ level 2 

(1-2 evaluations) levels 3&4 ( 1 evaluation) per year; 
Portfolios reviewed by placement committee. 

Other Key Components 

- Mandatory for new teachers 

- Downward movement possible 

- Appeal process by committee review 

- Sustained performance will allow teacher to remain at same level and step 

1986 Written Comments Analysis and Other Observations 
. - CL teachers evaluated and placed on first four levels 

- Concern that the key CLP evaluation criteria was the portfolio 

- Concern over the communication flow; a great deal of teacher input, but few clear 
cut decisions that were communicated on a timely basis 

- Concern over lack of inservice 

- District found it necessary to "cap" salaries due to budgetary requirements 

- A great deal of teacher resistance was felt as evidenced by petition sent to key 
CLP people 

- Stated that focus for 1986-87 will be a revised teacher evaluation and CLP 
placement. 
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according to a set of uniform time-lines has proven futile. This, in turn, revealed the folly 
of legislating a supposedly "fair," single set of such time-lines for compliance. Greater 
flexibility with respect to time-lines more closely conformed to the extreme diversity in 
readiness, and ironically it was more equitable as well. 

In contrast, there has emerged a consistent "bright spot" with regard to evaluating 
the reality of compliance with program designs and structures. Regardless of district 
readiness level, socio-economic status, or other individual differences, all teachers consider 
opportunities for professional improvement as a significant intrinsic motivator. They have 
availed themselves of inservice and other such incentives to the hilt. This initiative with 
respect to leadership and self-improvement has exemplified the finest qualities of the 
teaching profession and should serve as an ongoing source of pride for teachers and 
administrators alike. 

This combination of "best intentions 1 ' and diversity to implement them represented a 
formidable challenge to researchers and policy-makers. How could a well-developed 
external program, with lofty plans and ideals for educational reform, be successfully 
integrated into such a wide variety of organizational structures? 

The researchers responded to this challenge by stepping back and trying to identify 
the key factors which seemed to spell success or failure regarding external change efforts. 
Once these were pinpointed, an evaluation system could be devised to assess the actual 
"current level of operational health" of each of these critical areas within each district. As a 
result of this diagnostic procedure, both internal district decision-makers and external 
policy-makers would have a better gauge of the district's potential of success in actually 
following its roadmap of change effort successfully. 

The next section will describe a theoretical model of necessary factors which 
influence district potential for successful implementation of an external educational reform 
program such as Career Ladders. An associated diagnostic profiling procedure for 
assessing the operational level of each of these focus and support factors, as they currently 
exist within each district, will be explained next. Thr combination of this profiling 
procedure with careful consideration of accountability for program implementation has led 
to a multi-level placement procedure of applicant-districts - one which helps ensure a more 
realistic "match" of current capabilities and program participation requirements. Each of 
these areas will be discussed in turn. 
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District Readiness to Support Change: A Proposed 
Model of Kev Support and Focns Factors 

It quickly became apparent to the evaluators that the "uniform" legislative guidelines 
for program implementation, while no doubt designed to be clear, concise and 
straightforward, actually did not correspond well to the reality of participating districts' 
struggles to comply. That is, there existed an extreme diversity in individual districts' 
readiness to integrate and implement such external programs as Career Ladders in 
accordance with the single set of predetermined legislative time-lines. 

In an attempt to understand and account for this diversity, the researchers have 
developed a model of key focus and support factors which must be operating at initially 
satisfactory levels within a given district prior to its adoption of an external program of 
change. This model is graphically depicted in Figure 2, p. 7. 

The reader will note that external factors, such as legislative guidelines and state- 
mandated regulations and financing, form the foundation of the model. The within -district 
support factors include such areas as overall organizational climate and communication, as 
well as teacher-administrator evaluation procedures, professional input and ownership, and 
both intrinsic and extrinsic motivators. Especially noteworthy is the pivotal role played by 
professionals in addition to Career Ladder teachers themselves (for example, principals, 
superintendents, and school-board members) in determining the ultimate success or failure 
of the external program being implemented. At the pinnacle of the model one can find the 
essential focus factors, which include an adequately formulated and validated curriculum, 
ongoing opportunities for teacher skills development and leadership, and reliable 
procedures for linking teacher performance to student achievement using the latest available 
technology. 

P rofiling: A Methodology to Assess Individu al Districts' 
Readiness tn Supp ort Change Processes 

The aforementioned model has been repeatedly shown to be a valid indicator of 

those actual within-district elements which are pivotal to educational change and reform In 

particular, this model has resulted in the development of a unique "diagnostic" research 

procedure known as profiling. The purpose of profiling is to present, in convenient 

graphical form, the relative position (reflecting "current operational health") of all of the 

focus and support factors for a given district, as compared to one another and to a "zero 

baseline." Thus an evaluator can tell, at a glance, which individual factors are functioning 

most adequately (as pictured by their appearance and relative distance above this zero 

baseline), as well as those which are in critical need of attention and remediation (by their 

appearance and relative distance below the baseline). 



Figure 2 
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This factor-by-factor profile originated from the analysis of pilot-test districts' 
open-ended written responses to six questions. These questions asked all Career Ladder 
teachers, a random sample of non-Career-Ladder teachers, and all administrators to identify 
the areas of strength and insufficiency of the Career Ladder program overall, within their 
districts, and within their schools. 

These responses were read, summarized and clustered according to subjects' 
reference to particular areas of focus and support factors in their replies. For example, a 
mention of monetary incentives as a "strength" was keyed and clustered under the support 
factor entitled, "Extrinsic Motivators." Next, a weighting system was applied which 
essentially tallied the relative frequency of mention of each factor, with a "plus sign" 
assigned to mentions as strengths, and a "minus sign" for mentions as areas of weakness. 
Therefore, each focus and support factor for a profiled district yielded a net sum: the sum 
total of its mentions as an overall, district, or school strength, minus the sum of its 
mentions as an overall, district, or school weakness. (This procedure is akin to the 
accounting determination of "net income or loss" as "revenues" minus "expenses." If the 
former exceeds the latter, there is net income; otherwise, there is a net loss.) 

Figure 4, p. 11, illustrates the graphical profile of support factors for a 
hypothetical district. Note that four of the support factors for this district turned out to have 
a greater frequency of mention as strengths than as areas of insufficiency. That is, the net 
total for the four areas (strengths subtotal minus weaknesses subtotal) resulted in a positive 
number. That is why they are all depicted as being above the zero baseline (also known as 
"sufficiency of readiness baseline") in Figure 4, p. 1 1. In addition, the upper half of the 
chart (representing greater-than-zero scores) is therefore labeled as "Positive Readiness 
Factors" in the shaded triangle in Figure 4, p. 11. Specifically, these four areas are: 
Local Finance and Funding; Change and Improvement; Network; and finally, General 

Finance and Funding. 

Furthermore, of these four "relative strength" factors, Local Finance and Funding 
resulted in the highest positive point subtotal (shown as 30, in the bubble appearing in the 
lower-left-hand portion of its box). Thus, its relative ranking, in successively decreasing 
order of magnitude, is "1." This rank is depicted in the lower-right-hand box within the 
factor. Change and Improvement was the next-most-frequently mentioned relative 
strength, with an overall net total of 20 points and a relative rank of "2." The other two 
factors are interpreted similarly. 

On the other hand, the remaining six support factors received a proportionately 
greater frequency of mention as overall, district, and/or school weaknesses than a? 
strengths. That is why they appear below the sufficiency of readiness baseline in Figure 
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4, p. 11. For example, Professional Input and Ownership yielded a net score of -10. It 
can be seen, from glancing at the relative graphical array of "negative readiness factors" 
(shaded triangle in Figure 4, p. 11) that two of these support factors "tied" for being the 
greatest "anchors" or in most need of remediation prior to initiation of change efforts. 
These were Motivation: Intrinsic and Extrinsic and Organizational Factors (Climate and 
Communications), both of which resulted in a net score of -50. 

Figure 3, p. 9, depicts the four focus factors in the same relative graphical array. 
One can note here that Teacher Skills Development and Leadership was the sole focus 
factor which emerged as a relative strength; this outcome, incidentally, has been repeatedly 
found to hold true for all of the participating Career Ladders pilot-test districts. On the 
other hand, the remaining three focus factors are "anchors" to any well-intentioned attempts 
to apply and integrate an external teacher incentive and development program within this 
hypothetical district Of these three, Administrator Development and Leadership is most in 
need of urgent remediation, with a net score of -50. 

In summary, this profiling procedure has proven to be an easily understood and 
convenient way to assess individual districts 1 current readiness to support external change 
efforts such as Career Ladders. This type of information as to relative strengths and areas 
of insufficiency is useful in a number of key respects. For one thing, it can be used by 
legislators to help allocate a total dollar amount of available program funds more 
realistically and equitably, enabling a maximal "return on (public-dollar) investment" by 
ens -ring that a larger proportion of funds is awarded to those applicants with the greatest 
potential for success. In addition, it can serve as a useful diagnostic tool for individual 
districts in deciding whether they are, in fact, ready to apply for and initiate such external 
reform programs. Also, such profiling can serve as periodic "self-monitoring" of potential 
"red-flag" indicators of those areas which are "running smoothly," as well as those which 
are in need of immediate attention, once a given district has actually been approved for 
program funding. 

Combining Readiness with Accou ntability: 
Jhe Assessment Mod el of Alternative 
Program Placement Levels 

An important aspect of program implementation is an entity's ability to demonstrate 

accountability for tangible results. In other words, a school district needs to have in place 

reliable procedures for determining "where we've been, how we're doing and where we're 

going" relative to program objectives. With dependable self-monitoring feedback loops 

and reporting procedures in place, the participating entity can demonstrate its commitment 

to actualizing goals, including applying self-corrective procedures if necessary. 
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Accountability serves two major purposes: ensuring goal-oriented activity for external 
funding agencies and dependable, timely self -monitoring within the system itself. (This 
dual function is akin to the external and internal reporting mandates of financial accounting 
systems. The former fulfills the stewardship function for government agencies, lenders and 
shareholders. The latter includes self-contained information and feedback loops to 
managers, administrators and subordinates.) 

However, the important area of accountability, like other aspects of healthy system 
operations, is dependent to a large degree on overall system readiness. That is, a functional 
evaluation and reporting system cannot exist unless other components are prepared to 
support its creation. 

Accountability, in turn, affects assessment of overall system readiness to support 
change. The profiling procedure discussed in the immediately preceding section is a 
sophisticated mechanism for identifying those elements of an organizational system which 
are currently functioning at adequate levels to support integration of an external program 
such as Career Ladders, as well as those which are critically in need of remediation. Such 
a comprehensive self-assessment procedure depends on an accepted, reliable self- 
monitoring mechanism. 

This key interaction between readiness and accountability is therefore at the heart of 
individual districts' efforts to meet the stated objectives of an external program 
successfully. Diversities in maintaining this balance account, in large measure, for 
corresponding variation in program success. 

Figure 5, p. 13, depicts how this interaction may be combined with the profiling 
procedure in order to allow for multiple levels of districts' placement in the Career Ladder 
program. At the left-hand side of this model, one can see the same focus and support 
factors depicted in the readiness model of Figure 2, p. 7. The key factors which 
influence accountability are depicted in the upper portion of Figure 5, on the following 
page. In the uppermost boxes, the reader can see the external agencies (legislature, state 
board and department of education) to whom the aforementioned "stewardship" reporting 
responsibility is owed. The accountability function, in turn, is dependent upon the quality 
of system-specific evaluation procedures for its teachers and administrators. These should 
ideally be centered on the critical program goal of validly linking teacher performance to 
student achievement, in order to establish congruence of system activities with the overall 
objectives of the teacher incentive program. The ability of the district to develop such 
smoothly functioning evaluation systems depends upon such factors as teacher input and 
ownership, teacher development, and the present available level of finance and funding. 
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Figure 5 



Assessment Model for Projected 
District Readiness Levels 



STATE BOAHO OF EDUCATION ; 



DEPARTMENT OP EDUCATION 



/■Teacher J| 
P.-xtnership, 



Adminlstradve 



.Evaluation ... 

|.C^texk'fw.'; ; ; 
Advancement 



Student Achievement 
(Production A Outcomes) 



Teacher Skills 
Development & Leadership 



Administrator 
Development & Leadership 



Evaluation 
(Teacher /Specialist & Administrator) 



Motivation 
(Intrinsic/Extrinsic) 



X 



Local Finance & Funding 
(Salary Schedule) 



District R&D 
(Program Evaluation) 



Professional Input 
(Ownership) 



Program Designs 
and Structures 



Local Governing Board 
(Understanding & Support) 



Organizational Factors 
(Climate/Communication) 



v^;&udeot:J5 
Academic ' 
: Achievement.; 



■'- .' Teacher ' v $;i 
Dcvelopnienl 

|j | HI 




♦ ♦ ♦ ♦ 



Program 
Assurance of 
Accountability 




District Model 
Placement Level 

Based on 
Assessment and 
Profiled Needs 



Level I 



Level D 



Level IE 



Level HI 



Effective Schools 
Career Ladder Model 



Level II 



Transition Schools 
Career Ladder Model 



Level I 
Developing Schools 
Career Ladder Model 



1 



Application of 
Development 
& Improvement 

Models for 
Needed Change 



♦♦♦♦ ♦♦♦♦♦ 

ASSESSMENT & PLACEMENT" BASED ON . .. . " :^g§& 

AN D PROJECTED'EWE ANI> CHANGE BQUIREMENTS ' 



9 

ERIC 



13 



0 



These factors are ail depicted in the second row of boxes associated with the 
"accountability" function in the upper portion of Figure 5, p. 13. 

How can the existing diversities in district readiness and accountability be translated 
into maximally successful experiences 'nth external change processes such as Career 
Ladders? The diagnostic information obtained from the profiling procedure can be used to 
place applicants into one of three alternative levels of program participation. This process 
is diagrammatically shown in the right-hand portion of Figure 5, on the previous page. 
These levels allow for the best possible match between present district capabilities and 
corresponding program responsibilities, thereby allowing the applicant to "experience 
immediate success" as well as the necessary time to correct existing insufficiencies and to 
be prepared to progress to the next level. 

Each of these levels of program participation will be briefly discussed below. (The 
reader is referred to Summative Report IV, Pages 6 through 8, for additional detail.) 

The Developing Schools Career Ladder Model (DSCLM), or Level I, is designed to 
promote essential district growth in a wide variety of job enlargement assignments. Up to 
25% of teachers could be placed in Level I, and their assignments would include 
curriculum development, assessment of student learning and activities related to the unique 
social and demographic conditions faced by the district 

Upon successful attainment of these developmental objectives, the district could 
then apply for funding according to the Transitional Schools Career Ladder Model 
(TSCLM), or Level n. The developmental objectives of participants in the TSCLM would 
involve applying more technologically sophisticated procedures for accounting for student 
progress and linking it to teacher behaviors. Up to 50% of teachers could be recruited for 
program participation with Level I and II placement. 

The third level of district participation is known as the Effective Schools Career 
Ladder Model, or ESCLM. This level is reserved for those systems whose inteiTelated 
elements have been profiled as functioning at maximally effective operational levels, 
including a comprehensive evaluation system which is capable of providing reliable, timely 
information as to accountability in meeting program goals. 

This alternate-tier process of placement has been designed to enhance applicants* 
chances for success in meeting program objectives. In the first place, acceptance into the 
program is no longer "all-or-nothing." As a result, those districts with present deficiencies 
with respect to readiness and accountability can still qualify for participation if they show 
genuine promise for successful change. Thus they can experience some immediate 
successes with a set of objectives more carefully tailored to their current capabilities. These 
successes, in turn, provide the necessary intrinsic motivation to encourage system 
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improvement of areas of readiness and accountability, thereby strengthening the successful 
functioning of the overall organizational system, in addition to the more immediate goals of 
the external teacher-incentive program alone. As a consequence, the external change 
program is not only integrated more successfully; but it becomes a catalyst for needed 
improvements throughout the entire system itself. 

Summative Matrix Compar ison of Initial Legislative 
Mandates. Research Findings, and 
RHatffri Policy Implications 

The final portion of this document presents a side-by-side historical perspective on 
initial legislative regulations, along with directions for future program policy revision as 
suggested by summary research findings to date. This information is presented under 
headings which correspond to interrelated focus and support factors as depicted in the 
model in Figure 2, p. 7. Specifically, Exhibit A, p. 32-38, contain the sets of matrices 
which present this information for the focus and support factors, respectively. (Please 
note: in some cases, two or more components are combined, as they were jointly affected 
by legislation. An example of such overlap is in the two related areas of Student 
Achievement and Curriculum/Instruction/MeasuremenL) 

This matrix presentation thereby enables the reader to identify readily "where we've 
been and where we seem to be going" relative to the Career Ladders pilot-test program in 
the state of Arizona. By presenting historical and recommended policy implications side by 
side, it is hoped that the reader can therefore gain a more "holistic" perspective about the 
main areas of program impact and accomplishment, as opposed to "getting lost" in the 
sometimes-overwhelming and excessive detail of individual data and research findings. 

The next section contains qualitative, or "anecdotal," information from a number of 
participating districts about the specific effects of the Career Ladder Program on each of the 
focus and support factors identified in Figure 2, p. 7. These responses reflect their 
perceptions as to the actual impact of the pilot program on such important areas as student 
achievement, program designs and structures, and the skills development and leadership of 
teachers and administrators. These responses will be clustered into key themes under each 
focus and support factor subheading. In addition, illustrative quotes will be used 
throughout the discussion, in order to give the reader a flavor of the true effects of the 
program as actually experienced by these participants themselves. 
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Career Ladders Ane cdotal Information 

Introduction 

A comprehensive evaluation of the Career Ladder program would be incomplete 
without giving participants a chance to "tell the story in their own words." Qualitative 
(open-ended) data typically contain rich, detailed descriptions of program impact and 
results. These data also frequently go off in novel, surprising areas which may have not 
occurred to the external researchers. As a result, careful analysis of such open-ended 
responses provides a more complete and valid picture of the program as it was actually 
experienced by the participants themselves. 

These qualitative anecdotes were solicited from the members of the Career Ladder 
Pilot District Network (CLPDN) in September of 1989. Specifically, they were given a list 
of 24 focus and support factors and asked to write at least one positive comment 
concerning the effects of the Career Ladder program upon that area. 

Responses were received from seven of the 14 districts belonging to the CLPDN. 
As expected, they contain a variety of viewpoints, as well as some particularly colorful 
quotations concerning the perceived impact of Career Ladders. 

These main points are summarized in Exhibit B which appears on pages 39-46. 
They will also be briefly discussed under each focus and support factor subsection, directly 
below. 

Student Achievement (P roduction and Outcomes) 

In this area, the respondents pointed with pride to the percentage- and point-gains 
attained by their district's students on nationally normed tests such as the ITBS. They also 
indicated the hefty percentages of students and teachers meeting their targeted objectives for 
the school year. These results point to the beginnings of recognizing the need for 
accountability, as well as for linking teacher performance with student achievement in a 
valid and reliable manner. Obviously the existence of the CL program has spurred an 
awareness of the importance of a dependable ongoing evaluation system. 

One district referred to the increased emphasis on providing its personnel with the 
proper procedures for pre-post testing and assessing the associated magnitude of gain 
scores. Another pointed out that the student achievement component now constituted 40% 
of the point value of teachers' required action plans. 

In addition to improved documentation of gains in traditional areas, the CL program 
appears to have triggered a veritable "assessment versatility." One district noted, "Career 
Ladder has stimulated a move toward defining measurable outcomes in non-academic areas 
such as art, music and physical education." 
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Teachers and administrators also seemed more aware of the broader scope of the 
student-achievement assessment process. According to one district, M ... teachers are 
beginning to feel more comfortable dealing with outcomes as opposed to process." 

The following quote perhaps best captures teachers 1 feelings of accomplishment 
and self-pride resulting from learning "the state-of-the-art" technology in accounting for 
student achievement, as but one tangible benefit of the CL program. H 'I have been 
teaching 8 years and know that I am teaching better as a result of the Career Ladder ... and 
if that doesn't improve my students* learning, I don't know what will ... , M is one teachers 
explanation of student achievement gains/' according to one district respondent. 

Curriculum/In?;tTuction/Student Achie vement Measurement 

Five of the seven respondents described the tangible benefits of ongoing curriculum 
alignment. One district reported a 90% alignment in the basic skill areas, with related 
improvements in teaching strategies, specific content and classroom materials such as 
textbooks. Another stated, "Not only have we begun to make great strides in aligning the 
curriculum with the state's essential skills and norm referenced testing, but we have begun 
to develop criterion] referenced testing that is more reliable and valid for measuring student 
achievement." As with the previous area, a third respondent pointed out an increase in 
quantifiable objectives which are specifically targeted at improved instructional practices 

Above all, there appeared to be a heightened awareness on the part of both teachers 
and administrators of the pivotal role played by a soundly developed curriculum. " ' I do 
believe that having the [CL] Plan in the district has made a difference in the overall quality 
of instruction ... All teachers are paying more attention to curriculum planning and lesson 
development in a systematic way, ... " is how one administrator stated this belief," was the 
written comment received from one respondent. 

Teacher Skilk (Developm ent and Leadership^ 

Two general areas of satisfaction were evident in the responses to this topic. Career 
Ladders was perceived as fostering a greater number of high-quality formalized 
opportunities for ongoing improvement, such as inservice and peer coaching. The second 
frequently mentioned benefit of CL was a heightened sense of pride in one's image of 
professionalism, particularly with an increased number of genuine teacher leaders. 

Not only are there increased growth programs; teachers themselves have realized 
corresponding opportunities to assume major leadership roles in these, and similar, 
activities. More and more inservice is now being conducted by a district's own staff. 
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This feeling of accomplishment which accompanies leadership has successfully 
been applied to other areas. Such shared responsibility for decision-making, according to 
one respondent, allow teachers " ... to gain a perspective and appreciation for the 'big 
picture. 1 " Another administrator observed that: " ... the CL promotes the sharing of 
leadership among administrators and teachers ... that's what research says ought to be 
happening in schools ... and in our district, it is." 

This satisfaction with increased teacher-leadership has been matched by the 
intrinsically motivating sense r ' professionalism. One CL teacher commented, "I am proud 
to have participated [in the Plan] ... and feel more professional as a result/ 1 Another wrote, 
"The level of professionalism in the classroom has never been more refined" 

Administrator Skills (Development a nd Leadership^ 

Respondents repeatedly referred to the improved leadership and enhanced skills 
which their administrators have acquired in evaluative activities generally, as a direct result 
of the CL program. Specifically, administrators have received ongoing inservice and other 
educational opportunities designed to enhance their abilities in teacher evaluation and 
student achievement gain-score assessment. One district has instituted continuing inservice 
for its administrators in these areas, with the targeted goal of attaining a 0.95 inter-rater 
reliability coefficient in teacher evaluation. Another provided a variety of staff- 
development-type educational opportunities for its principals, with such featured topics as 
the correlates of Effective Schools. 

As with the preceding section, CL has engendered a greater amount of shared 
decision-making as a result of administrative awareness and support of the program. Such 
joint efforts at shared responsibility have, in turn, fostered an increasing sense of 
satisfaction, as well as camaraderie between superiors and their subordinates. One district 
refers to the joint CL program administration and the "non-adversarial model" of its 
associated "meet-and-confer process" for teachers and their principals. Another admitted 
that, "There has been a partnership developed that no one felt [was] possible in the past." 

Along these lines, CL was credited with creating a bolstered awareness on the part 
of administrators of "what's going on in the classroom" on a day-to-day basis. One 
principal noted, "I feel that the best way to provide support for teachers is to see them in the 
classroom. I can be much more helpful as an instructional leader when I am more aware of 
the dynamics of the class." 

Despite such increased opportunities for direct administrator involvement, some 
untapped areas still remain. One district observed that, while administrators* support for 
the CL program was admittedly high, they presently had little or no chance of getting 
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involved in the actual implementation of the program, as " ... our CL plan is teacher-run." 
Nonetheless, an increasing awareness of administrators' natural curiosity and sincere desire 
for greater hands-on involvement can serve as a catalyst for greater collaboration in 
decision-making with the clear potential for such eventual benefits to administrators as 
those described by other districts above. 

Evaluation (T eacher and Administrator') 

As in the preceding section, respondents focused on increased cooperation, 
consensus and shared responsibilities between teachers and administrators as beneficial 
"by-products" of the Career Ladder program. One commented: "There has been more 
cooperation between teachers and administrators on evaluation in the last five years than 
anyone would have thought possible." The increased reliance on peer evaluation in another 
participating district has resulted in " ... eliminating any conflict with the building 
principal's time and/or authority." A third respondent observed: "The evaluation 
instruments for Career Ladder teachers were mutually developed, with a majority of teacher 
input, by teachers and administrators utilizing a 'consensus' decision making process. 
There is [a] positive working relationship between teachers and administrators with regard 
to the evaluation process, as evidenced by no career ladder appeals of evaluation 
outcome[s] by teachers." 

These increased collaborative efforts have also resulted in a heightened awareness 
on the part of principals of day-to-day classroom management activities. One secondary 
principal commented: "Through the combination of the CLAI review process and 
evaluations, I am exposed to a lot more of my teachers' work than in the past." This 
hands-on involvement of their supervisors, in turn, has certainly not gone unnoticed by the 
teachers themselves. One Career Ladder teacher noted: "1 think my principal is more aware 
of the breadth and depth of my curriculum since spending more time in my room ... " 

Other tangible benefits of the CL program consist of more objective and specific 
evaluation procedures, as well as increased provision for administrator training in the 
evaluation process. One district responded that: "Administrators have been trained more 
thoroughly in teacher evaluation and have the confidence of their staff when it comes to 
making summative judgments on teacher performance." Finally, another respondent noted 
that the CL program had spurred necessary revisions targeted at improvement in the 
district's existing evaluation process. 




Motivation qn trinsic/Rxtrinsid 

The Career Ladder program has clearly fostered a widespread sense of self-pride 
and self-esteem in its teacher-participants. Primary among these sources of intrinsic 
satisfaction is an increased perception of professionalism. As articulated by one 
respondent: "It's rewarding to be part of a plan in which teachers are definitely recognized 
for their efforts that go 'above and beyond' ... The Career Ladder system has helped me 
professionally and has made me value being treated as 'a professional.' " Another noted 
that 'Teacher empowerment through increased role efficacy has resulted in increased 
positive attitudes." 

Two districts pointed out the interactive nature of extrinsic and intrinsic rewards 
regarding this heightened sense of teacher professionalism. As explained by the first, "We 
will be rewarded (monetarily and otherwise) for being 'accountable.' I call this forced 
professionalism." The second stated that, "At thp outset of the program, the primary 
motivation may have been monetary, but it became obvious that money would be a short 
term motivator. In the last three years the primary motivating force has been to become 
more professional as a teacher and to have more success with students." 

In addition to the satisfactions derived from increased professionalism and 
accountability, several additional intrinsic motivators were mentioned by members of the 
CLPDN. These included increased opportunities for professional growth; a more precise 
goal orientation in one's activities; and the increased emphasis on student achievement 
itself. 

With regard to extrinsic motivators, both the provision of monetary incentives and 
the increased opportunities for financial advancement were cited. One district also referred 
to the beneficial effects of paying CL teachers to attend workshops and other training 
sessions, pointing out that this compensation policy resulted in 422 out of 491 eligible 
teachers attending a recent mini-grant workshop series. 

I,ocal Finance and Fund ing fSalarv Scheduled 

A mixed bag of issues appeared under this particular heading. One which was 
singled out by two of the respondents was the supplementary nature of the local salary 
schedule. According to the first of these, "We use the Career Ladder funds to supplement 
the difference between where Career Ladder teachers would be on Framework and where 
they are on CL." The second noted, "The district has always supplemented state funding 
for our career ladder program and will continue to do so in the future. The salary schedule 
is based on performance in the classroom and other additional responsibilities are primarily 
paid for through addendums and are not part of the teachers' career ladders salaries." 
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Two other districts stressed the fact that the vast majority of the total allocated CL 
<^fund does, indeed, go directly to teacher salaries and benefits. A full 93% in fact, is 
'directly applied toward teacher compensation in one of these districts. 

Another issue discussed in this section is the explicit endorsement of teachers of the 
performance-based nature of the local salary system. Another district pointed out that there 
had been beneficial effects on its teacher recruitment and retention as a direct result of its 
CL salary plan. 

District Research and Development (Program E valuation^ 

It is truly encouraging to note the rapid growth of local research efforts in response 
to the central issues touched upon in the Career Ladder program. Four of the seven 
respondents described studies which focused on student achievement and related outcomes. 
One administrator, for instance, recently completed a doctoral dissertation dealing with 
student achievement, teacher morale, and teacher turnover. Another district has created a 
comprehensive computerized data base which has been specially designed to generate an 
informational and diagnostic "student achievement profile." 

In addition, districts have installed periodic research-based reporting mechanisms, 
similar to management information systems for chief executives in the private sector. One 
respondent described a series of advisory council sub-committees to perform in-depth 
evaluations of various aspects of the Career Ladder program. Another pointed out that 
district administrators were required to report research results periodically to the steering 
committee. 

Another district has now instituted a practice of sending out periodic surveys to 
gauge "what's working." Finally, there was a greater interest in keeping up with other 
state and national research studies, not only to keep current but to apply the conclusions 
and implications to one's own program, in order to improve its functioning if possible. 

Professional Input (Ownership) 

Most respondents described the significant teacher and administrator representation 
on steering committees which had been enhanced in terms of their operational 
effectiveness* Two common themes emerged in their responses: the genuine feelings of 
teacher ownership of the program, and related intrinsic satisfaction. One district noted, 
"Because the steering committee is teacher-run, the career ladder teachers have developed, 
revised and taken responsibility for career ladder. This has produced a great deal of 
ownership." (underline in original text) The resultant feelings of self-satisfaction and 
accomplishment in having one's input taken seriously is well illustrated in the following 
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comment from one administrator: "Basically, I think the Career Ladder Committee has 
done a top-notch job of producing a plan we can be proud of and implementing it 
effectively." 

Districts shared in detail the composition and tasks of career ladder steering 
committees. The ongoing nature of participant input was most evident from their written 
responses. Teachers and administrators had formalized means of periodic, cyclical 
program review, input and evaluative feedback. As one district put it: " ... an ongoing 
professional advisory committee ... [which is] made up of teachers and administrators ... 
meets no less than one time per month and any concerns relating to career ladder are 
discussed at these meetings. The district office also tries to meet with each individual staff 
[member] at each school at least twice a year to discuss the entire program." Some specific 
program areas which teachers and administrators have significantly influenced by their 
continual input include budget setting and revision, staff development activities, and design 
of the evaluation systems themselves. 

One district effectively summed up its staffs enthusiasm about their perceived 
degree of professional input and how this change was attributable to the Career Ladder 
program. "We are using a feedback form to get specific personal data from teachers about 
how they have been impacted by CL programs responses are very enthusiastic. More 
people want an opportunity to become involved with the legislative process - do whatever 
they can to keep the program." 

Program Designs and Structures 

Five of the seven respondents praised the clear focus on instructional excellence 
and student achievement which Career Ladders has brought to their educational systems. 
"Our career ladder program is designed around instruction and those factors that directly 
enhance students' success. Additional responsibilities relate directly to instruction," 
according to one district. A career ladder teacher in another district observed that, "It [the 
CL program] has made me a great deal more organized, do a better job of teaching skills 
sequentially, and has encouraged me to challenge my special education students with higher 
order thinking skills." 

This emphasis on top-notch instructional processes has also fostered corresponding 
improvements in teacher cooperation. Another teacher noted, " ... We continue to share 
ideas, but now we also share approaches to the CLAI process or ideas about how to keep 
records or document our lesson plans." Thus the effects of sharper educational program 
focus have begun to be felt throughout the professional, interpersonal climate as well. 
Other tangible intrinsic benefits have included increased innovation and time expended on 
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improved instructional activities, particularly in those instances where related financial 
incentives are explicitly built into the district's plan. 

Awareness of the need to construct a workable program plan has also extended into 
responsible financial stewardship on the part of participating districts. That is, they are 
properly concerned about attaining measurable benefits, in the form of improved instruction 
and related gains in student achievement, in return for program dollars invested. One 
respondent wrote, "We are looking closely at budget relative to program benefits and 
degree of involvement in order to use funds for the greatest value to the most teachers." 
Another observed that programs were now being explicidy designed with "teachers' needs 
in mind." 

Local Governing Board ^Understand ing and Support 

A common theme of responses to this sectidn was the tremendous supportiveness 
of their Board members for the Career Ladder program >nd its related educational 
objectives. One person wrote, "Our local governing board has, from the outset, supported 
the career ladder concept and has provided the necessary monetary support to supplement 
state funding. They have a very good understanding of the career ladder concept and 
specific meetings have been held to update them on the career ladder program." In the 
words of another district: " The ... board unanimously supports the District's career ladder 
program, as evidenced by its public action of approving the program with a five to zero 
vote of support." 

The importance of educating Board members with regard to the goals and inner 
workings of the Career Ladder program has been recognized by the participant-districts. 
Two of them mentioned specific inservice provided to the school board on this topic. 
Increased awareness and support, in turn, have translated into overt financial backing of the 
program. (One exception of "lip service" was lamented by one respondent as follows: " 
[The] Board supports CL in theory, but has not committed themselves with matching funds 
or/extra' monies.") 

Perhaps one answer to this dilemma is an increased communication among 
teachers, administrators and school-board members with regard to the educational 
objectives of teacher-incentive programs such as Career Ladders. As one district pointed 
out, "Our Board is supportive of Career Ladders because our superintendent is so positive 
and involved in the Plan." 
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Organization al Factors (Climate and Communications^ 

Respondents eagerly pointed with pride to the satisfactions of improved channels of 
communication and information access which were attributed to the Career Ladder 
program. Specifically* there was a tremendous satisfaction with "being heard" by one's 
superiors, which in turn was effectively translated into more active involvement in program 
goals and activities on the part of teachers. 

One district noted that, "There has never been more dialogue between 
administrators land teachers than we now have with the Career Ladder project Overall, the 
climate in the district is very good because teachers know that they always have access to 
information and the administrative network," 

These opportunities for upward and downward communication in the organizational 
hierarchy have increased in variety and scope since the inception of the the Career Ladders 
program in some districts. As one pointed out: "Teachers involved in the Career Ladder 
Plan generally report that having the Career Ladder Plan in the district is beneficial to 
students, teachers, and schools. Teachers have had many forums to express their opinions 
and feelings about the Career Ladder Plan: meetings, written surveys and narratives, 
interviews, verbal contact with representatives, and input into the Far West Laboratory 
report completed in June, 1989." 

This realization of "finally having been heard," and the accompanying sense of 
professionalism, has inspired teacher-participants to translate their new-found satisfactions 
into increased involvement. "There is a very positive professional climate as well as 
effective communication in place between teachers, administrators and members of the 
Governing Board. This is primarily a result of having a standing career ladder committee 
on the program which is representative of teachers and one administrator." According to 
another respondent, "As a result of a CL newsletter, CL bulletin boards in every school, 
and the efforts of CL council representatives, teachers seem to be more informed about the 
plan, and consequently [they] respond more positively and participate to a much greater 
degreein theoptions." 

A final tribute to the positive effects of the Career Ladder program upon 
organizational climate is, in effect, its "successful minimization of the negative." No 
innovation, however appealing and/or well-intentioned, is without its critics. However, 
one district pointed out that even potentially disruptive effects of this group are virtually nil 
when it comes to Career Ladders. M ... others (opponents) [are] just resigned to [the] fact 
that CL is here to stay." Such non-destructive tolerance may well serve as evidence of an 
organizational climate which is able to manage conflict and support a multitude of different 
points of view among its members. 
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rhanpe and Improveme nt fLonp-Ranpe Plans') 

A wide variety of observable benefits were cited by participant districts as being due 
to the positive influence of the Career Ladder program. Once again, the increased long- 
term emphasis on linking teacher performance to improved student achievement received 
multiple mentions. One district even singled out its efforts to acquire and apply the latest 
"technology" in the assessment and measurement process. The district is continuing to 
strive to update information regarding reliable indicators, that measure teachers' success 
and our long-term goal is to continue to strive for reliability and validity in the area of 
teacher evaluation and student achievement." Another district mentioned that serious 
efforts were now being made with respect to long-range curriculum alignment 

Such long-term strategic planning relative to improved student achievement has 
been accomplished by pooling the talents of several levels of personnel in the organizational 
hierarchy. That is, the practice of shared decision-making once again came in for favorable 
mention. One respondent commented, "Teachers and administrators, from the Plan's 
conception, continue to have input to the committee to help refine and improve the Plan." 
As a result, participants have realized the long-run planning benefits of improved channels 
of communication and more timely, accurate information flow which have already been 
refeiTed to in the section on organizational climate. 

As in the private sector, operating conditions are seldom perfectly predictable. A 
flexible approach to the strategic planning process which includes alternative 
"contingency" subplans is actually the most realistic route to take. One respondent 
commended this equivalent approach in educational planning: "Our Career Ladder plan has 
proven to be adaptable to the needs of our teachers. This flexibility is a source of strength 
for our program." 

Finally, participating districts are f eeling sufficiently satisfied with their long-range 
planning successes to begin to consider innovative ways in which they can document and 
share these planning experiences with colleagues. One respondent described a video which 
his/her district is preparing and which will deal with its Career Ladder program 
experiences. 

District Readiness Leve l rAssessmenf) 

The responses to this section overwhelmingly confirmed the need for other essential 
organizational elements to be in place, and functioning satisfactorily, prior to initiation of an 
external change effort such as CL. That is, the focus and support model of readiness 
factors appears to hold true as reflected in participants' experiences. 
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District representatives spoke of the strong need for ongoing staff development 
programs, curriculum alignment, and active involvement of administrators in effecting 
positive change. One person wrote, "The district has been successful with the program 
because many elements such as leadership, communication, staff development, curriculum 
development, research development, etc., ... either were in place prior to our career ladder 
program or were developed shortly thereafter. Our organizational climate ... was an 
essential element for our success with the program." Another observed that: The 
emphasis on student achievement, curriculum alignment and research all are critical to the 
District's eligibility for the 'Expert' level." A third respondent felt that: "Because 
administrative support is high and we have a staff development plan already well in place, 
[we] adapted easily to Career Ladder and [will] continue to experience success." 

One can conclude from such comments that "the total system" critically needs to be 
in harmony for an outside teacher incentive program to have the greatest potential for 
achieving its goals. Ongoing monitoring of the individual elements of this system are 
wairanted in light of maximizing their positive "meshing" with well-intentioned external 
programs, in order that all components continue to run smoothly. 

Essentia] Elements for Educational Improvemen t and Success 

Judging from their responses, participating districts have realized a noticeable 
increase in the quality of such factors as focus on student achievement and teacher 
accountability. Along these lines, they have noted a greater sharing and communication 
among teachers; more application of innovative teaching techniques; a greater variety of 
ongoing staff-development opportunities; and enhanced support from the top. 

One respondent wrote, "Although the processes of planning, implementing and 
evaluating the areas of instruction, classroom management and counseling are imperative, 
the bottom line is student achievement. The combination of both process and product 
activities will result in the improvement of education and student success." Another 
expressed the opinion that, "Communication and leadership are the key elements for 
educational improvement A willingness to trust one another and work together have made 
our career ladder successful." 

State Finan ce and Funding 

Comments for this factor centered around the critical need to supplement local 
funding with state support. In addition, two respondents praised the sufficiency of the 
funding provided by the state for their respective districts' local Career Ladder programs. 
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Two others believed that the funds which are provided are distributed in an equitable 
manner* 

One individual felt that, "State funding for the career ladder program has certainly 
made the job of implementing the program much easier. The graduated increase of state 
funding over the life of the pilot program is a sound idea and should be incorporated into 
future plans for implementation." 

Another indicated the pivotal role played by state-level financial support in the 
attempts to have program implementation reach its noteworthy goals. "Without state 
funding Career Ladder Plans would have been extremely difficult, if possible at all to 
formulate. The amount of money provided, has made it possible for teachers to realize a 
significant value placed upon the excellent teaching they do ... We're dependent on state 
funding to perpetuate the Career Ladder Plan in its present form, so that it can continue to 
benefit students, teachers and schools." 

State Governing Board (Accountabilit y/Implementation^ 

Several comments illustrated the increased level and quality of communications 
between state and local agencies in joining forces to attain the objectives of the Career 
Ladder program. In particular, the exemplary leadership of the State Superintendent was 
singled out for special praise. 

As one district representative put it: "Although, the State Board of Education may 
not know all of the details about career ladder programs, the district has been fortunate that 
the State Superintendent, Diane Bishop, nas been assigned as a legislative representative to 
oversee the District's career ladder program. Briefing sessions have been conducted with 
State Superintendent Bishop and representatives of the district's career ladder program. As 
a result, this relationship and ongoing line of communication is evident between the 
district's career ladder program and office of the State Superintendent for public 
instruction." According to another, "The state Governing Board's visit (with a legislative 
representative) each year has. allowed us a chance to explain our plan and demonstrate its 
effectiveness district- wide." 

Legislative Guidelines 

Though it may seem paradoxical at first glance, the "rules and regulations" were 
applauded for both their specificity and their flexibility. That is, the participating districts 
valued the overall structure and detail evident in the initial legislation. However, they also 
appreciated the fact that there was sufficient "room" in the law for individual district 
diversity to be incorporated. 




One person pointed out: "Legislative guidelines have been broad enough to let 
districts develop creative Career Ladder Plans that allows for the unique needs of individual 
districts. These guidelines give us latitude with funds which allow districts alternative 
compensation plans/' According to another district representative, "The guidelines were 
feasible enough to allow for local initiative in designing and implementing [them]/' A third 
noted: "Legislative guidelines currently in effect are sufficiently structured, so that 
individual districts can meet the needs of their staff" Finally, there was this statement: 
"The legislative guidelines for phase three school districts are very clear. It is without 
reservation that [our] career ladder program has been very effective and efficient as a result 
of the specific guidelines provided by the legislature." 

Another district singled out the guidelines as having the 'proper focus' on student 
learning and teacher accountability, which have always been the overall intent of the Career 
Ladder incentive program. "Legislative guidelines have been reasonable for a pilot 
program and the emphasis on student achievement and teacher evaluation is right on 
target." 

The last noteworthy point regarding legislative guidelines is their inherent "freedom 
of choice" with regard to Career Ladder participation. "The guideline that states voluntary 
participation by districts in the Career Ladder Plan is viewed by this Career Ladder 
Committee to be a valuable concept that should be protected." 

Analysis of Research V ariables. Program 
^valuation and Imp rovement 

Participating districts repeatedly cited the usefulness of the various research efforts 
in providing needed and practical information. Results of locally conducted studies have 
been helpful in assessing student achievement; predicting student learning for the long term; 
overall needs assessment; and improving CL program implementation. 

In the words of one respondent, "[Our] School District continues to collect student 
data on both an aggregate and disaggregate basis. The District will continue to work 
towards improvement of the career ladder program evaluation. Specifically, [we are] 
utilizing longitudinal data on student achievement and determination of expected growth for 
all students assigned to career ladder teachers." Another observed, "The research that 
Northern Arizona University has done has been extremely helpful in evaluating the success 
of our career ladder program. This research also provides valuable information such as the 
readiness levels that can be helpful in the future implementation of career ladder programs." 
The most colorful way of conveying the practical application of research results is 
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illustrated in the following quote: "Now [there is] a mountain of data useful in diagnosing 
local needs." 

The following comment perhaps best illustrates the clear value that districts perceive 
in continuing their "home-grown" research projects. "The analysis of career ladders has 
stated what we have seen - students arc benefitting, teachers are working and career 
ladders are successful in solid districts." 

Arizona Legislature 

The legislature was commended for the quality and frequency of its communication 
efforts with individual districts. This "hands-on" approach to monitoring the effects of the 
CL program was recognized and appreciated. 

One district representative wrote, "The direct involvement by the Arizona 
Legislature in the career ladder program, we thipk, has been extremely helpful and 
enlightening. It has been exciting to have direct access to legislators and explain our 
positions relative to the pilot program." According to another, "[Our] School District 
maintains a communication network not only with the assigned legislators who oversee the 
District's career ladder program, but all legislators who are in the legislative district ... arc 
kept informed. A major task for the school district will be to increase each legislator's 
knowledge to a higher level with regard to the District's CLP." 

The strong leadership which the Arizona legislature has taken with regard to the 
goals of career ladders is clearly evident to the participating districts. One person noted, 
'The Arizona legislature is to be commended for taking leadership in passing the Arizona 
Career Ladder Pilot Program. Education is becoming more excellent in Arizona, for 
students, because of improved instruction developed through the Career Ladder Plan. It is 
also a strong factor in attracting and keeping excellent teachers in Arizona. This plan needs 
to be continued as standards in education continue to rise to the demands of the 1990's. A 
Career Ladder teacher states, ' ... Frankly, I would be devastated if a Plan that has proven 
so beneficial - to me, to my students, to my school suddenly disappeared/ " 

The legislature was also commended for the overall support which it has provided 
to participating districts. One individual pointed out that "[It] gave the [CL] project the staff 
support and technical support (NAU) required to succeed." 

Career Ladder Pilot District Network ( CLPDN . 

Participating districts praised the network for serving as an effective ongoing 
mechanism for information dissemination, team approaches to problem solving, and high- 
quality communication. One person responded, "The Career Ladder Network has provided 
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an avenue for districts to communicate with each other and the Legislative Education 
Committee. Benefits derived from the Career Ladder Network are: increased 
understanding of the philosophy and goals of other districts with Career Ladder plans, and 
an opportunity to discuss problems and suggestions. The network meetings have also 
allowed districts to work together in compiling a report commenting on Career Ladder 
districts' attitudes on the future of Career Ladders in Arizona." Another felt that, "The 
Career Ladder Network Committee has been a very helpful resource in the implementation 
phase of career ladders. The sharing of ideas has made the implementation less stressful 
and, in my opinion, more productive." According to a third, "This is a sufficiently 
organized network that has been helpful in keeping the pilot districts 'in touch' and in 
keeping the legislature informed about Career Ladders." Finally, a fourth individual 
summed up the preceding comments quite well with: 'Terrific sense of communication and 
sharing." 

Department g £ FHn^tinn fPORUState Board 

Only one response was received to this section. The subject referenced his/her 
answer to "State Governing Board (Accountability)," in which the state board was 
commended for improving the overall level of communication, as well as the commitment 
to quality of the current State Superintendent of Education. 

Summary Comments 

It seems clear from the preceding responses that participating districts have had very 
positive experiences with the Career Ladder Program. They have applauded its goals of 
greater teacher effectiveness and accountability for increased student achievement. In 
addition, they have recognized the effects of Career Ladders upon the individual 
organizational focus and support factors (illustrated in Figure 2 on p. 7). Implementation 
of this program has greatly enhanced teacher professionalism, overall communications, and 
involvementof administrators in attaining the laudable goals of this program. In addition, 
there has been greater cooperation between participating districts and external agencies such 
as the legislature and the department of education. Pilot districts are reaching out to one 
another to share ideas, troubleshoot as a team, and generally promote the key goals of the 
program. They are eagerly initiating their own research efforts, as well as learning from 
studies conducted by others. Systems have been strengthened, both from within and 
without, by the focused efforts of Career Ladders to attain greater student learning. It is 
encouraging to note these positive responses to an external change program, and therefore 
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to realize that educational professionals will apply themselves to change efforts with such 
zeal and enthusiasm. 

Conclusion 

Many valuable lessons have been learned from the five-year pilot-test of the Arizona 
Career Ladder teacher incentive and development program. The evaluation process has 
revealed some startling discrepancies between the initial legislative mandates and the reality 
of districts' attempts to comply. Foremost among these was an extreme diversity with 
respect to individual districts' readiness to conform to uniform legislated regulations and 
time-lines. Therefore, the well-intended "fairness 11 and "simplicity" of a single set of 
guidelines resulted instead in some accidental inequities for those districts struggling to 
comply with requirements beyond their present readiness capabilities. 

In their attempts to understand this discrepancy between intent and reality, the 
evaluators developed a comprehensive theoretical model of focus and support factors which 
must be functioning adequately prior to successful initiation of change efforts. This 
proposed theoretical model has since been repeatedly shown to be a valid and accurate 
depiction of actual organizational processes for participating districts. Therefore, it has 
been adopted as part of a unique profiling procedure, whereby each individual district (both 
as an applicant and as an accepted program participant) can be objectively assessed with 
respect to the current "level of operational health" of each focus and support factor. As a 
result, both program decision-makers and district administrators obtain valuable feedback 
as to possible necessary overall remediation for any of their system elements. 
Furthermore, future legislation concerning continuations and/or extensions of Career 
Ladders can be revised to correspond more closely to the reality of program 
implementation. 

In coming "full circle" from monitoring of present legislation to making policy 
recommendations based on systematic observation of program participants, the evaluation 
process has contributed a significant explanatory model of organizational factors affecting 
system readiness for change. The benefits realized from applying this model have clearly 
extended beyond the pilot-test of the Career Ladder program itself. Administrators, 
researchers and legislators have gained a more thorough and accurate understanding of the 
essential system-specific factors which can determine the ultimate success or failure of any 
externally developed reform movement, however well-intentioned. This allows for a 
realization of greater cost efficiencies and corresponding returns on investment of scarce 
public funding of such projects. In addition, the diagnostic information provided by the 
profiling process serves as a useful overall evaluative model for district decision-makers. * 
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EXHIBIT A 



ARIZONA CAREER LADDER RESEARCH & EVALUATION PROJECT 

Matrices of Legislative Mandates, Research Findings and Policy 
Recommendations Related to Essential Organizational 
Focus and Support Factors 
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Table 1 A Matrix of Legislative Mandates, Research Findings and Policy Recommendations Related to 
Essential Organizational Focus Factors 



Student Achievement & Curriculum/Instruction/Measurement 


Legislative Mandates 


Research Findings 


Policy Implications 
& Recommendations 


Document how the participating CL 
district's plan is designed to 
improve student achievement 


1) Teachers have realized signifi- 
cant gains with respect to their 
professional skills development 
& their impact on improved 
student achievement 
Futhermore, three studies have 
statistically demonstrated this 
link between levels of teacher 
performance & student 'gain 
scores on a comprehensive 
standardized achievement test, 
(See Exhibit B, p. 48) 

2) Most districts cannot tangibly 
demonstrate progam's effects 
on student achievement at the 
present time. Deficiencies exist 
with respect to: 

a) Establishing a valid 
curriculum; 

b) Relating process evaluation 
to proouci eviaUaiion, 

c) Developing teacher compe- 
tencies to pre- & post- test; 

d) Reliably linking teacher per- 
formance to student achieve- 
ment, via correcdy normed 
local, state & national tests. 


a) Recognize diversity in readiness 
to demonstrate accountability 
for student learning by assess- 
ing & placing individual dist- 
ricts on one of three alternative 
models of program participa- 
tion: 

Districts would progress from dev- 
eloping schools to transitional 
schools to effective schools, by 
developing the four capabilities 
listed under point 2 (Research 
Findings), thereby reliably linking 
teacher performance to student 
achievement; 

b) Explicitly include "educational 
specialists" (e.g. school 
psychologists, counselors, & 
librarians) within the staff 
Qeiiniuon oi pruicaMunaia wnu 
significantly impact student 
achievement; 

c) Encourage periodic review & 
revision of curricula by local 
universities; 

d) Encoun-ge partnerships with 
private sector (business & 
indusny) to work on educat- 
ional systems, structures & 
basic procedures of operation. 
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Table 1 A Matrix of Legislative Mandates, Research Findings and Policy Recommendations Related 
Essential Organizational Focus Factors 



Administrator Skills Development & Leadership 


/ 

Legislative Mandates 


Research Findings 


Policy Implications 
& Recommendations 


Explicit provision was made for 
administrative involvement with 
respect to two key areas: 

a) Majority support of adminis- 
trative well as teacher, 
personnel by confidential ballot 

j is required prior to program 
adoption; and 

b) Documentation of procedures 
for support of faculty develop- 
ment & evaluation of principals 
is also required. 


Administrative personnel have 
consistently expressed a desire to 
be more actively involved with 
planning & implementation of the 
Career Ladder program. At the 
same time, however, they feel that 
they have not been given adequate 
inservice & necessary batkground 
information on the Career Ladder 
program. Furthermore, they have 
expressed a pervasive feeling of 
being "left out" of significant 
Career Ladder decision-making 
activities, such as strategic program 
planning, implementation & 
evaluation. 


Increased communication & shared 
decision-making responsibilities 
among administrators, Career 
Ladder program directors & 
participating teachers, including: 

a) Mandated inservice & other 
educational opportunities for 
administrators in Career 
Ladder applicant districts, as to 
specifics on program intent & 
purpose; 

b) Clearly delineated, ongoing 
opportunities for district 
administrators to become more 
actively involved in such 
pivotal Career Ladder related 
decision activities as evaluation 
of Career Ladder teachers. 
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Table 1. A Matrix of Legislative Mandates, Research Findings and Policy Recommendations Related 
Essential Organizational Focus Factors 



Teacher Skills Development & Leadership 



Legislative Mandates 


Research Findings 


Policy Implications 
& Recommendations 


Job enlargement via additional 
skills & responsibilities is required 


Delegation of a wider variety & 
scope of duties to teachers would 
alleviate pressures on administrators. 
Teachers can also provide valuable 
input relative to such tasks as 
curriculum development, mentoring, 
:>eer coaching & clinical supervision. 


a) Continue "additional respons- 
ibilities" legislative require- 
ment along with explicit recog- 
nition of the extended time re- 
quired for such duties (e.g., 
contract extensions for top- 
level teachers akin to those in 
the private sector); 

b) Incoiporate increasing variety 
& scope of such activities as 
part of districts 1 progressions 
through the three alternative 
models of CL participation 
(e.g., curriculum development 
assignments at developing 
school levels, vs. mentoring 
duties at effective school 
levels); 

c) Offer "educational specialists" 
the same opportunities for 
job enlargement as those 
awarded to other teachers; 

d) Review present teacher-certi- 
fication requirements to deter- 
mine the adequacy for licensure 
of high-quality instructional 
personnel. 



ERIC 



35 

42 



Page 1 of 3 



Table 2 A Matrix of Legislative Mandates, Research Findings and Policy Recommendations Related 
Essential Organizational Support Factors 



Professional Input and Ownership & Organizational Climate 


Legislative Mandates 


Research Findings 


Policy Implications 
& Recommendations 


Evidence of teacher consensus 
with respect to program approval 
& development of plan is 
required. 


Those programs which have been 
implemented in cooperative environ- 
ments with well-functioning chan- 
nels of communication have also 
been the most successful. Corro- 
boration of organizational 
psychology finding from the private 
sector "People will support that 
which they help create" (e.g., 
teacher representation on steering 
committees wnicn nave uccn 
successfully integrated with the 
balance of the district's admini- 
strative decision-making structure). 


a) Majority approval by confi- 
dential ballot should continue 
to be a prerequisite for pro- 
gram implementation & 
continuation; 

b) Expand legislated teacher role 
beyond merely that of 
"consultant" to meaningful 

& periodic input into all 

j»QTV*rtQ frf* nroPTam ITTTnlc- 

mentation & refinement and; 

c) Work for improvements of 
dysfunctional aspects of 
existing organizational climate 
(e.g., excessive competitive- 
ness & withholding of infor- 
mation) prior to implemen- 
tation of external programs 
such as Career Ladders. 
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Table 2 A Matrix of Legislative Mandates, Research Findings and Policy Recommendations Related to 
Essential Organizational Support Factors 

Program Designs and Structures 



Legislative Mandates 


Research Findings 


Policy Implications 
& Recommendations 


Requirement for individual district 
plans to contain specifics with 
regard to: 

a) Professional advancement; 

b) Education as a requirement; 

c) Teacher improvement; 

d) Plans for program imple- 
mentation; 

e) Procedures for periodic 
teacher evaluation; and 

f) Procedures for periodic 
administrator evaluation. 


a) Most districts have well devel- 
oped program designs & struc- 
tures. However, extreme 
diversities exist with respect to 
readiness for implementation of 
these plans. Such differences in 
readiness have led to difficulties 
in compliance with legislated 
uniform time lines for change; 

b) Increased focus on improvement 
of teacher performance has led 
participants to avail themselves 
of inservice opportunities. 
However, others have not 
moved beyond randomly accum- 
ulating college credits & similar 
activities. 

c) Local efforts at periodic program 
evaluation have increased; s 

d) Those districts which have made 
efforts at increasing direct admin 
istrative involvement in the pro- 
gram design & implementation 
process have also experienced 
the most success with their 
Career Ladder programs. 


a) Continue requiring applicant 
districts to submit well-docu- 
mented program designs & 
structures; 

b) Build in graduated & flexible 
time lines, to correspond to 
individual differences with 
respect to current readiness 
for implementation; 

c) Continue to require provision 
of opportunities for improve- 
ment of performance; 

A\ DAnnirp evaluation Rj t^vipu/ 
U ) IxCUUXXw CValUaliUIl vX iwviww 

by internally developed inte- 
grative research & develop- 
ment component; 

e) Continue administrator eval- 
uation concerning their 
knowledge & extent of 
Career Ladder support; and 

f) Develop evaluation systems 
for "educational specialists" 
similar to those for other 
teachers. 
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Table.2. A Matrix of Legislative Mandates, Research Findings and Policy Recommendations Related to 
Essential Organizational Support Factors 



Local Governing Board 



Legislative Mandate 


Research Findings 


Policy Implications 
& Recommendations 


Provision for governing board 
to apply for Career Ladder pro- 
gram participation has been made. 


I./v*al Povftrninp hoard has consis- 
tently been shown to be a critical 
component in determining ultimate 
success or failure of Career Ladder 
program implementation. In par- 
ticular, potential for success has 
been considerably enhanced in 
those districts with boards who are 
informed of Career Ladder program 
intent & who do not use it as a 
vehicle to further their o\frn separate 
political agendas. 


Mandate assessment of govern- 
ing board by third-party eval- 
ation, with respect to its level 
of understanding & support 
of Career Ladder agency 
concepts; 

b) If necessary, provide in- 
service to increase such under- 
standing & support to accept- 
able levels. 


Professional Networks 


Legislative Mandate 


Research Findings 


Policy Implications 
& Recommendations 


No explicit legislative require- 
ments exist at present. 


Network has served as a valuable 
liaison between districts & policy- 
making bodies (e.g., by generating 
recommendations for legislative 
program revision). 


Require all funded Career Ladder 
districts to join and participate in 
a network which functions inde- 
pendently of formal governing 
boards. 


State Governing Board (ADE) 


Legislative Mandate 


Research Findings 


Policy Implications 
& Recommendations 


None exist at present. 


Ongoing research studies & contin- 
ued interest in teacher incentive 
plans by ADE have been docu- 
mented. In particular, continue 
research on linking teacher perfor- 
mance to student achievement 
reliably & devising comprehensive 
& valid curricula. Implementation 
& validation of Arizona Student 
Assessment Plan should remain a 
top priority. 


Formulate policy guidelines to 
encourage Career Ladder districts 
to adopt the comprehensive ADE 
curriculum. Also, encourage 
individual districts to assist ADE 
in ongoing validation of the 
assessment plan. 
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EXHIBIT B 



ARIZONA CAREER LADDER RESEARCH & EVALUATION PROJECT 
SUMMARY OF POSITIVE ORGANIZATIONAL COMPONENTS: 

Summated Matrices Depicting Positive Anecdotes Related to 
Interrelated Organizational Focus & Support Factors 
In Successful Career Ladder Pilot-Test 
School Districts 
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Summary of Key Points: Career Ladder Anecdotes 



Student Achievement 
Production & Outcomes 


Curriculum/Instruction/ 
Student Achievement 
Measurement 


Teacher Skills 
Development & Leadership 


* Increased ability to document 
pre- and post- tests, and to 
assess associated gain scores. 

* Increased ability to define 
measurable outcomes in 
traditionally "hard-to-quantify" 
areas (e.g. art, music, phys/ed). 

* Greater emphasis on student 
achievement documentation in 
teacher's action plans. 

* Increased documentation of 
standardized test results. 

* Greater teacher satisfaction, 
self-pride, and sense of 
accomplishmerit relative to 
student achievement gain-score 
assessment. 


* Tangible, ongoing curriculum 
alignment with district objectives 
(5). 

* Creation of locally developed 
assessment tools. 

* Increased focus on higher quality 
content, skills,classroom .materials 
and instructional strategies. 

* Heightened teacher and 
administrator awareness of overall 
importance of sound curriculum 
development. 


* Emergence of "teacher leaders" (4) 

* P.jreased professionalism (2). 

* Improved peer coaching (2). 

* Year-round in service opportunities 

(2). 

* Teacher skills development with 
respect to: 

a) planning 

b) instruction 

c) interpersonal skills 

d) human growth & development. 
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Table 3 Continued 



Summary of Key Points: Career Ladder Anecdotes 



Administrator Skills 
Development & Leadership 


Evaluation 
(Teachers & Administrators) 


Motivation (Intrinsic 
& Extrinsic) 


* Improved leadership skills 
in: 

a) teacher evaluation (3) 

b) student achievement 
assessment (3) 

c) instructional skills 

* Ongoing inservice oppor- 
tunities for administrators. 

* Shared decision-making 
(teachers & administrators) 

* Heightened administrator 
awareness of day-to-day 
classroom activities. 


* Greater sense of cooperation (3) 

* Better evaluation training for 
administrators (2). 

* Teacher input into evaluation 
process (2). 

* More objective & specific evalu- 
ation procedures. 

* Heightened administrator aware- 
ness of day-to-day classrooni 
activities. 

* Initiation of necessary revisions 
in local evaluation system in 
process. 


Intrinsic: 

* Perception of greater piofessionalisn 
(3). 

* Increased opportunities for 
professional growth (2). 

* Increased sense of accountability (2) 

* Improved goal orientation. 

* Increased emphasis on student 
achievement. 

Extrinsic: 

* Existence of greater financial 
incentives (2). 

* Rapid monetary increases possible. 

* Compensation for attendance at 
workshops and other teacher- 
improvement activities. 



4i 48 



Page 3 of 7 
Table 3 Continued 



Summary of Key Points: Career Ladder Anecdotes 



Local Finance 
& Funding 
(Salary Schedule) 


District Research 
& Development 
(Program Evaluation) 


Professional 

Input 
(Ownership) 


Program Designs 
& Structures 


* Majority of funds going 
for teacher compensation 
(2). 

* Separateness of Career 
Ladders compensation 
funding from standard 
salary schedule (2). 

* Movement toward parity 
(more competitive). 

* Equitable funding dist- 
ribution. 

♦Effectiveness of local 
salary schedule in 
teacher recruitment & 
retention. 

♦Teachers have endorsed 
a performance-based 
approach to salaries. 


♦Comprehensive internal 
research & evaluation in: 

a) student achievement 
(4) 

b) staff development (2) 

c) instructional practices 

d) teacher morale & 
turnover. 

★Development of comput- 
erized personnel & 
student data bases. 

♦Establishment of advisory 
council sub-committees. 

* Conducting own local 
experimental studies. 

* Reporting of research 
results to steering 
committee. 

* Ongoing review of other 
related state & national 
studies. 


* Ongoing mechanisms for 
continual teacher & 
administrator input (4). 

* Feelings of genuine 
ownership (2). 

* Tangible effects realized 
with respect to: 

a) budget 

b) staff 

c) development 

d) design, review, & 
revisions of 
evaluation system 
itself. 


* Focus on excellence in 
instruction & student 
achievement (4). 

♦Innovation is encour- 
aged. 

♦Monetary rewards for 
expenditure of teacher 
time & effects. 

♦Financial "stewardship 1 
(concern for realizing 
maximal benefits from 
program funds spent). 

* Designed "with 
teachers 1 needs in 
mind." 

* Commitment to 
professionalism. 

* Improved effectiveness 
in: 

a) planning 

b) implementing 

c) classroom manage- 
ment skills 

d) counseling skills. 
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Table 3 Continued 



Summary of Key Points; Career Ladder Anecdotes 



Local Governing Board 
(Understanding & Support) 


Organizational Factors 
Climate & Communications 


Change & Improvements 
(Long-Range Plan) 


* Supportive attitude of Board 
towards Career Ladder 
Objectives (3). 

* Active desire to leani more 
about Career Ladders (2) 

* Importance of provision of 
financial support by Board (2). 

* Enhanced interpersonal relation- 
ships & communication 
between Board members & 
administrators. 


* Improved top-down & bottom-up 
channels of communication (4). 

* Sense of "being listened to". (3) 

* Provision of timely, relevant, 
dependable information (2). 

* Satisfaction with shared decision 
making (2). 

* Pride in one's professional image. 

* Better management of conflict 


* Greater emphasis on student 
achievement (2). 

* Shared decision-making 
responsibilities (2). 

* Concern for teacher skills, 
improvement & evaluations (2). 

* Improved overall organizational 
structure. 

* Support from administrators. 

* Support from school board. 

* Focus on cunicular alignment. 

* Initiation of locally planned & 
developed research activities. 

* Greater flexibility in district 
planning process. 

* Improved validity & reliability 
of measurement & assessment 
process. 

* Provision of timely, relevant, & 
dependable information. 

* Wider dissemination of local 
Career Ladders program 
experiences. 
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Table 3 Continued 

Summary of Key Points: Career Ladder Anecdotes 



District Readiness 
Level (Assessment) 


Essential Elements for 
Educational Improvenrcnt 
& Success 


Professional 
Networks 


State Finance 
& Funding 


* Other organizational 
factors working in 
harmony: 

a) staff development (2) 

b) well-aligned curriculum 
(2) 

c) support from 
administrators 

d) well-developed training 
procedures 

e) well-developed 
evaluation system 


* Greater accountability. 

* Focus on student achieve- 
ment. 

* Improved teaching skills. 

* Increased sharing of ideas 
among teachers. 

* Better overall communi- 
cation. 

* Better leadership. 

* Increased opportunities for 
staff development & 
training. 


* Cooperation vs. 
competition. 

* More sharing. 

♦Administrator involve- 
ment. 

*Wider networking 
(with other existing 
organizations). 

• 


* Critical need for state- 
level funding (2). 

* Adequacy of amount of 
state-level funding (2). 

* Equity in disbursement 
of funds (2). 

* Approval of built-in 
graduated increases. 
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Table 3 Continued 



Summary of Key Points: Career Ladder Anecdotes 



State uoverning 
Board (Accountability 
& Implementation) 


Legislative 
Guidelines 


Analysis or Kesearcn 
Variables, Program 
Evaluation & Improvement 


Arizona 
Legislature 


* Improved communi- 
cations (2). 

* Quality State 
Superintendent. 


* Sufficient structure 
& detail (2). 

* Flexibility with 
regard to local 
diversity (2), 

* Proper emphasis 
on student achieve- 
ment. 

* Allowance for 
voluntary particip- 
ation. 


* Generation of useful 
information for. 

a) assessing current 
student achievement; 

b) forecasting future 
learning gains; 

c) needs assessment; 

d) program implemen- 
tation. 

* 


* Ongoing commun- 
ication with districts 
(3). 

* Quality leadership. 

* Provision of adequate 
support services. 
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Table 3 Continued 

Summary of Key Points; Career Ladder Anecdotes 



Career Ladder Pilot 
District Network (CLPDN) 


Department of Education 
& State Board 


* Sharing ideas (3). 

* Better communication (3). 

* Better information dissemination (2). 


* Improved communications. 

* Quality State Superintendent 
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ftllMM ATTVF. REPORT IX 
EXECUTIVE SUMMARY: 

Recommendations to the JLCCL for Pre-Legislative 
Phase of the Arizona Career Ladder Program. 
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Northern Arizona University 



CENTER FOR EXCELLENCE IN EDUCATION 

EXECUTIVE SUMMARY: RECOMMENDATIONS TO THE JLCCL 
FOR PRE-LEGISLATIVE PHASE OF 
THE ARIZONA CAREER LADDER PROGRAM 

THURSDAY, NOVEMBER 30, 1989 

Introduction 

Northern Arizona University, in cooperation with Arizona State University and the 
University of Arizona, has been researching the Career Ladder concept for the past tive 
years. Our major conclusion is that such a program do£l positively impact student 
achievement by its effects on motivation & retention of high-quality teachers. Other key 
findings appear below. 

Summary Findings 

1. The researchers have pinpointed "what works & what doesn't" for program success. 
That is, we've learned how school organizations function, as well as which elements of 
these organizations directly affect school reform. In short: we know the essential 
elements of success, not just in terms of organizations, but all the way from legislative 
structures through policy formulation. We have developed a model which shows all of 
these factors & how they affect rr.form movements. 

2 The researchers need to keep studying those ingredients of schools & districts which 
can "make or break" well-intentioned reform movements. How are they operating in 
Career Ladder districts & how do they affect program success or failure? 

3. The researchers also can use what we know about "success elements" to help the 
"have-not" districts improve & restructure, so that they can participate in this reform 
program and experience supportive change. In this way, all districts in the state can be 
helped to ensure (irrnuntahility for student prowess. 

4. Programs such as Career Ladders require sufficient funding to ensure their successful 
integration into districts. 

5. Two critical lessons have been learned from observing CL program operation in 
Arizona & other states: 

a) Such programs have tended to revert to solidified bureaucracies which stifle the 
dynamics & creativity upon which reform movements have been built; 

b) where special-interest & outside political interference has taken over, program 
results have been sabotaged . Specifically, there has been information filtering & 
distortion of frva|narinn results. 
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6. An objective, outside "3rd-partv evaluator" would help prevent such negative results. 
This external assessment dement should be written into legislative policy & should also 
continue to be completely free of special-interest interference : If this "3rd-party 
evaluator" has an opportunity to look at the total structure of participating districts, & 
can freely make objective recommendations for any needed adjustments in the system, 
these reform programs can, & will, achieve the positive results on teachers & students 
which the legislation intended. 

Concluding Comments 

Northern Arizona University has had a solid commitment to evaluating program results for 
the entire five years of the CL pilot-test program. We are now carefully studying the 
impact of the political & special-interest influence upon the carefully designed goals of such 
reform movements. Currently the research is focusing on the part of the model which 
relates to the legislative process phase of political policy. In addition, we are continuing to 
study the "success" & "failure" ingredients of school organizations & how they influence 
the program. 

Within the next few weeks, we will have available a highly readable report of summary 
conclusions, recommendations, and a report of "positive anecdotes," or successful 
experiences with the CL program as reported to us by the participating districts. 

Our complete results are always available to any & all interested parties. Please feel free to 
contact Dr. Richard D. Packard, Manager of the Arizona Career Ladder Research & 
Evaluation Project, with any questions or concerns at (602) 523-5852, or at the address 
shown on page 46. 
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EXHIBIT D 



^T M TUATTVF REPORT X 
THE ARIZONA CAREER LADDER RESEARCH & EVALUATION PROJECT 
FINDINGS RELATED TO STUDENT ACHIEVEMENT: 
Qualitative & Quantitative Evidence Relative to the Positive Impact 
of the Career Ladders Teacher Incentive Program on 
Student Achievement in Arizona 
Pilot-Test School Districts 
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^CoiicJiisions 



Helmstadter/Walton-Braver 
(ASU) — ► 



Fimbres, Sunnyside 
(UofA) 



Nine, Apache Junction 
(NAU) — 



Dickson/DeGracie/Guy 
(Mesa) 



The Network 



Amphitheater, Apache 
Junction, Catalina Foothills 
Cave Creek, Creighton, 
Dysart, Flowing Wells, 
Ganado, Kyrene, Litchfielc 
Mesa,Peoria, Sunnyside, 
Window Rock 



Packard/Dereshiwsky 
(NAU) 



After implementation of fee Career Ladder Teacher Incentive Program, 
there was a 150% increase in Phase I school districts* impact 
son student achievement {ovemon-CL districts in the state) on 
a nationally normed standardized basic skills test of reading, language 
& math ability. ■ ■■■■■ ■ 1 . [ " ■ "__ 



L Student performance cm a nationally normed test of reading, 
;■• . language, & math skills was higher in classes taught by CL 
teachers. • 

2. Number of years of teacher experience by itself is not related 
to student achievement • •• ; _ - ■ ;: • • - : 



Students taught by CL teachers who are pl^xd higher on the ladder, 
achieve at a higher level on nationally normed tests of reading, -\ 
language, & math skills than do those placed lower on the ladder. 



L After just one year of the CL.program, class averages in . 
reading, math, & language on a nationally normed test: 

a) were higher for CL teachers than for non-CL teachers; 

b) were higher for CL teachers placed higher on the ladder. 

2. Number of years of teacher experience by itself is not related to 
student achievement. ; ; 



Some factors reported by districts which are positively related to student 
achievement, and which are due directly to their participation in the CL 
program, are: 

1. Better curriculum alignment; 

2. Higher-level learning objectives being set by teachers; 

3. Increased "hands-on" involvement of their administrators in 
■day-to-day classroom teaching activities; 

4. More locally developed learning methods & materials; and 

5. More communication & sharing among teachers. 
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CL teachers are learning to calculate their own gain scores on a 
variety of national & locally developed tests which directly show 
their accountability for student achievement 
In addition, districts are learning how to show their actual impact on 
student achievement statistically, using scores from national, state & 
local tests. • 

CL teachers are documenting student learning which is happening in 
their classrooms, as well as their own accountability for student • 
achievement, in their portfolios. 

The expected future student achievement can be predicted for any 
type of subject taught & any type of test. 

: Case studies & other qualitative research has validated the need for : 
restructuring of certain pans of the organization so that they are better 
ifocused on the major goals of teacher development & student achieve- 
ment; i.e., teacher evaluation & development procedures which are more 
directly tied to those activities which actually affect student learning. 



Dr. Richard D. Packard, Manager, The Arizona Career Ladder Research & Evaluation Project, 
Center for Excellence in Education, P.O. Box 5774, NAU, Flagstaff, AZ 86011, Ph: (602)523-5852 
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* Students in Phase I CL districts are reaching their learning 
potentials at a dramatically higher level than they were prior to 
organizational restructuring & implementation of the CL pro- 
gram, and they are impacting student achievement even more when 
they are compared to non-CL districts. 



* Students taught by CL te?chers are making learning gains which 
are statistically signficant. 



■ ■:>:■:■:■. 



* Student achievement depends more on the level of teacher 
performance than on the number of ye ars of teaching. 




CL teachers understand their local, district, and classroom 
curriculum objectives better than they did before; they are 
receiving better inservice which is directed toward improved 
teaching effectiveness; and they are able to see greater student 
learning in subject(s) they are teaching. 



■mm 



• : ": -: • ' 

mmW& 



mmmwmmwmmm^ 

:. \.V-.>: : - >::.'.■.••■•■•. \ ■. - 



mmm 



CONCLUSIONS 

t 

Impact of Career Ladders on Student Achievement 
In Arizona Pilot-Test Districts 



ERIC 



52 59 



